R Notebook

UKDS logo

Guide to Interactive Visualisations

In this guide, you’ll be shown how to make 4 key interactive visualisations, which include:

• Basic bar chart
• Grouped bar chart
• Stacked bar chart
• Scatterplot
• Scatterplot + dropdown menu
• Line graph

To create these visualisations, we’ll be using the ‘plotly’ package.

Datasets used in this workshop are from the 2021 UK census, and involve the new voluntary question which focuses on gender identity. In particular, we explore the relationship between age and gender ethnicity, as well as ethnicity and gender identity.

Let’s get started by importing the necessary packages.

NOTE: If you’re not following along with Binder, and you have your own computational environment, make sure you install the necessary packages through the command line before proceeding to import.

Install packages

Uncomment the lines below to install the packages if you’re not working in Binder.

# install.packages("readr")
# install.packages("dplyr")
# install.packages("stringr")
# install.packages("shiny")
# install.packages("ggplot2")
# install.packages("plotly")

Load in packages

# Allows us to read-in csv files
library(readr) 
# For data manipulation
library(dplyr) 

Attaching package: ‘dplyr’

The following objects are masked from ‘package:stats’:

    filter, lag

The following objects are masked from ‘package:base’:

    intersect, setdiff, setequal, union

# For regular expression operations 
library(stringr) 
# library(shiny)
library(ggplot2)
# Used tp create interactive visualisations
library(plotly)

Registered S3 method overwritten by 'data.table':
  method           from
  print.data.table     

Attaching package: ‘plotly’

The following object is masked from ‘package:ggplot2’:

    last_plot

The following object is masked from ‘package:stats’:

    filter

The following object is masked from ‘package:graphics’:

    layout

Dataset 1

The first dataset that we’ll be focusing on is a really simple dataset which shows the total counts for 8 gender identity categories across England and Wales. We’ll do a bit of data cleaning, remove unnecessary categories (such as ‘Does not apply’), and then calculate the % of each gender identity category. Then, we’ll create a simple interactive bar chart which displays the percentage by gender identity category, whilst enabling some interactivity when we hover over each bar.

# Load in dataset

df <- read_csv('../Data/GI_det_EW.csv')

Rows: 8 Columns: 5
── Column specification ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Delimiter: ","
chr (3): England and Wales Code, England and Wales, Gender identity (8 categories)
dbl (2): Gender identity (8 categories) Code, Observation

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

# Brief glimpse of data structure
# But can also click on the dataset in the Environment pane
head(df, 10)

# Let's check out the dimensions
dim(df)

[1] 8 5

Data cleaning

• Clean column names
• Filter out unecessary categories
• Clean gender identity category values - too wordy
• Ensure gender_identity column is a factor with levels in desired order

# str_replace_all() method finds all substrings which match the regex and replaces them with empty string
# First, let's replace any brackets with empty strings
colnames(df) <- str_replace_all(colnames(df), "\\s*\\([^)]*\\)", "")

# Lowercase column text and replace empty spaces with "_"
colnames(df) <- tolower(colnames(df))
colnames(df) <- str_replace_all(colnames(df), " ", "_")

# Let's see if it worked..
colnames(df)

[1] "england_and_wales_code" "england_and_wales"      "gender_identity_code"   "gender_identity"        "observation"           

Pipes and other operators..

So, we’ve already come across the assignment operator ‘<-’ which is used to assign a value. E.g. df <- read_csv(‘Data/GI_age.csv’), here we assign our csv file to a dataframe variable called ‘df’.

But, we’re now going to encounter the pipe operator ‘%>%’ which can seem intimidating at first but is actually pretty simple. It’s used to pass the result of one function directly into the next function. E.g. df <- df %>% filter(gender_identity_code != -8), here we start with our df and pass it to the filter function using the pipe operator. This basically supplies the filter() function with its first argument, which is the dataframe to filter on. And here we encounter a logical operator ‘!=’ within the filter() function, which specifies that we should only keep rows where gender_identity_code is not equal to -8.

Dollar sign operator - $

This operator is used to access elements, such as columns of a dataframe, by name. Below, we use it to access the gender identity code column, where we want to view the unique values.

# Get rid of columns that do not apply
df <- df %>% 
  filter(gender_identity_code != -8) 

# Use unique and access column to output its unique values

unique(df$gender_identity_code)

[1] 1 2 3 4 5 6 7

# Let's take a look at our unique values in our gender_identity category column

unique(df$gender_identity)

[1] "Gender identity the same as sex registered at birth"                                  
[2] "Gender identity different from sex registered at birth but no specific identity given"
[3] "Trans woman"                                                                          
[4] "Trans man"                                                                            
[5] "Non-binary"                                                                           
[6] "All other gender identities"                                                          
[7] "Not answered"                                                                         

# Use combo of mutate and recode to replace multiple values in column
# .default ensures that any value not matching those specified are left unchanged

df <- df %>%
  mutate(gender_identity = recode(gender_identity,
    "Gender identity the same as sex registered at birth" = "Cisgender",
    "Gender identity different from sex registered at birth but no specific identity given" = "Gender identity different from sex",
    .default = gender_identity))

# Let's see if it worked...
unique(df$gender_identity)

[1] "Cisgender"                          "Gender identity different from sex" "Trans woman"                       
[4] "Trans man"                          "Non-binary"                         "All other gender identities"       
[7] "Not answered"                      

# We use factor to convert gender_identity column to a factor with specified levels 
# This tells Plotly the exact order in which to display categories

df$gender_identity <- factor(df$gender_identity, levels = c(
  "Cisgender",
  "Gender identity different from sex",
  "Trans woman",
  "Trans man",
  "All other gender identities",
  "Not answered"
))

Question

How is gender identity represented across England and Wales?

Some subquestions that this can help us answer:

• Which gender identity category is most prevalent?
• Which gender identity is the least prevalent?

Data pre-processing

Before we can plot our data, we need to calculate the percentage of each gender identity category. The mutate() function adds a new column ‘percentage’ to df, and applies the following calculation to each row.

# mutate() is used to add new variables to a df or modify existing ones
df <- df %>% mutate(percentage = round(observation / sum(observation) * 100, 2))

# Let's take a look..
head(df$percentage)

[1] 93.46  0.24  0.10  0.10  0.06  0.04

Basic interactive bar chart

Now we can create our first simple interactive visualisation. To do so we use Plotly’s plot_ly function, and supply the parameters with the necessary arguments. You’ll notice that we use the tilde operator (~) quite a bit when building our graph. By preceding relevant variables with ~ it tells R to look for that variable within the dataframe.

# Create the bar chart visualization with percentages on the y-axis
fig <- plot_ly(data = df, x = ~gender_identity, y = ~percentage, type = 'bar',
               # defines how the bars should be styled
               marker = list(color = 'rgb(158,202,225)', line = list(color = 'rgb(8,48,107)', width = 1.5)),
               width = 800, height = 600)

# Let's check it out
fig

Warning: Ignoring 1 observations
Warning: Ignoring 1 observations

Using layout() method

Once a graph has been created, we can use the layout method to customise the appearance and layout. This allows you to modify things such as titles, legend details, axis properties, etc, without needing to recreate the figure from scratch.

# Let's apply a log scale to our y-axis so this graph is easier to interpet 

fig <- layout(fig,
              title = 'Percentage of Each Gender Identity in England and Wales',
              # set showline to true, otherwise it disappears when we apply log scale
              xaxis = list(title = 'Gender Identity', showline = TRUE),
              yaxis = list(type = 'log', title = 'Percentage (Log Scale)'))

fig

Warning: Ignoring 1 observations
Warning: Ignoring 1 observations

Tooltips

When using different R libraries that are geared towards interactive visualisations, you’ll often come across ‘tooltips’. These are small boxes that provide information when a user hovers over a part of a data visualisation such as: a point on a graph, a bar in a bar chart, or a segment in a pie chart. They are used to display additional information about the data point or object, providing more context without cluttering up the chart. In Plotly tooltips are referred to as ‘hover_data’.

All interactive plotly graphs come with default hover data, so when you scroll over a bar or a scatterplot data point it will display the specific x-axis value and y-axis value. But, variety is the spice of life and there’s going to be times when you want to leverage this feature to include interesting info that isn’t included by default. For instance, for our bar chart, I’d like to add in data from the ‘Observation’ column, which shows the raw count for each gender identity category.

To do this it’s quite easy. We use the text and hoverinfo parameter in the plot_ly function, with text defining the variables we’d like to include and how they should appear, and hoverinfo ensuring that this text is displayed in the tooltips. So, let’s create the graph again, but this time let’s specify our tooltips.

new_fig <- plot_ly(data = df, x = ~gender_identity, y = ~percentage, type = 'bar',
                   # ~paste  combines multiple pieces of text and data into one string
               hovertext = ~paste("Gender: ", gender_identity, 
                                  #<br> is HTML code for a line break
                                  # sprintf - used to format strings
                             "<br>Percentage: ", sprintf("%.2f%%", percentage), 
                             "<br>Observations: ", observation),
               # tells plotly to only display the text provided in hovertext
               hoverinfo = 'text',
               marker = list(color = 'rgb(158,202,225)', line = list(color = 'rgb(8,48,107)', width = 1.5)),
               width = 800, height = 600)

# Apply a log scale to the y-axis
new_fig <- layout(new_fig,
              title = 'Percentage of Each Gender Identity in England and Wales',
              xaxis = list(title = 'Gender Identity', showline = TRUE),
              yaxis = list(type = 'log', title = 'Percentage (Log Scale)'))

new_fig

Warning: Ignoring 1 observations
Warning: Ignoring 1 observations

Dataset 2

This dataset classifies residents by gender identity and age, with the unit of analysis being England and Wales.

# Load in dataset 

df2 <- read_csv('../Data/GI_age.csv')

Rows: 42 Columns: 7
── Column specification ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Delimiter: ","
chr (4): England and Wales Code, England and Wales, Gender identity (7 categories), Age (6 categories)
dbl (3): Gender identity (7 categories) Code, Age (6 categories) Code, Observation

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

# Brief glimpse of data structure
head(df2, 10)

# Let's check out the dimensions

dim(df2)

[1] 42  7

Data Cleaning

• Clean column names
• Filter out unecessary categories
• Clean gender identity category values - too wordy
• Ensure gender_identity column is a factor with levels in desired order
• Clean age category values - too wordy

We’ll whiz through this, because it’s the same stuff we did for the last dataset.

# str_replace_all() method finds all substrings which match the regex and replaces them with empty string
# First, let's replace any brackets with empty strings
colnames(df2) <- str_replace_all(colnames(df2), "\\s*\\([^)]*\\)", "")

# Lowercase column text and replace empty spaces with "_"
colnames(df2) <- tolower(colnames(df2))
colnames(df2) <- str_replace_all(colnames(df2), " ", "_")

# Let's see if it worked..
colnames(df2)

[1] "england_and_wales_code" "england_and_wales"      "gender_identity_code"   "gender_identity"        "age_code"              
[6] "age"                    "observation"           

# Get rid of columns that do not apply
df2 <- df2 %>% 
  filter(gender_identity_code != -8) 

# Use unique and access column to output its unique values

unique(df2$gender_identity_code)

[1] 1 2 3 4 5 6

# Get rid of redundant age category
# Further filter data
df2 <- df2 %>%
  filter(age_code != 1)

# Clean up the values in the 'age' column. Let's shorten them.

# Chain str_replace() calls together to apply multiple string replacements in succession
# Each str_replace() call is applied to the result of the previous one
df2$age <- df2$age %>%
  str_replace('Aged ', '') %>%
  str_replace('to', '-') %>%
  str_replace('years', '') %>%
  str_replace('and over', '+') %>%
  str_replace(' - ', '-')

# We can pass our df to the select function, where we specify the column we're interested in.
# Then, we pipe the output to the head function.
df2 %>%
  select(age) %>%
  head()

# Use combo of mutate and recode to replace multiple values in column
# .default ensures that any value not matching those specified are left unchanged

df2 <- df2 %>%
  mutate(gender_identity = recode(gender_identity,
    "Gender identity the same as sex registered at birth" = "Cisgender",
    "Gender identity different from sex registered at birth but no specific identity given" = "Gender identity different from sex",
    .default = gender_identity))

unique(df2$gender_identity)

[1] "Cisgender"                          "Gender identity different from sex" "Trans woman"                       
[4] "Trans man"                          "All other gender identities"        "Not answered"                      

# We use factor to convert gender_identity column to a factor with specified levels 
# This tells Plotly the exact order in which to display categories

df2$gender_identity <- factor(df2$gender_identity, levels = c(
  "Cisgender",
  "Gender identity different from sex",
  "Trans woman",
  "Trans man",
  "All other gender identities",
  "Not answered"
))

Question

How is gender identity distributed among different age groups?

Some subquestions that this can help us answer:

• What % of trans women are aged 16-24 years?
• Are older age groups over represented in the ‘non-response’ category?

Data pre-processing

Calculate percentages

Below, we use the group_by function to group the data by ‘gender_identity’ and calculate the percentage within each group. Then the mutate() function adds a new column ‘percentage’ to df, which (for each group) divides the observation by the sum of observations, multiplies it by 100, and rounds it up to 2 decimal points. We then use the ungroup function when we’re done with the grouping operation.

df2 <- df2 %>%
  group_by(gender_identity) %>%
  mutate(percentage = round((observation / sum(observation) * 100), 2)) %>%
  ungroup()

head(df2)

Interactive grouped bar chart

When creating grouped bar charts, there’s a few subtle differences that you’ll need to account for in the code. First, we’ll need to make sure this is a grouped bar chart, which we can set with the ‘barmode’ parameter. Second, we’ll need a way to colour each bar in each group, according to age categories, which we can do with the ‘color’ and ‘colors’ parameters.

# Create a grouped bar chart with hover information
fig2 <- plot_ly(data = df2, x = ~gender_identity, y = ~percentage, type = 'bar',
               # color specifies which variable to colour by
               # colors specifies the colour palette to use, and how many colours are required
               color = ~age, colors = RColorBrewer::brewer.pal(length(unique(df2$age)), "Set2"),
               hoverinfo = 'text',
               hovertext = ~paste("Observation: ", observation,
                                  "<br>Percentage: ", sprintf("%.2f%%", percentage),
                                  "<br>Age group: ", age),
               marker = list(line = list(color = 'rgba(255,255,255, 0.5)', width = 0.5)),
               width = 800, height = 600)
               

fig2

fig2 <- layout(fig2,title = 'Distribution of Gender Identity Categories Among Age Groups',
         xaxis = list(title = 'Gender Identity'),
         yaxis = list(title = 'Percentage'),
         legend = list(title = list(text = 'Age Group')))

fig2

Stacked bar chart

The method I show below simply converts the previously made grouped bar chart ‘fig2’ to a stacked bar chart. Stacked bar charts can only be created using the layout() function to change the barmode, as the default is a grouped bar chart.

# Convert to stacked bar chart

st_fig <- layout(fig2,
         barmode = 'stack')

st_fig

Dataset 3

This dataset classifies residents by gender identity and ethnic group, with the unit of analysis being the 331 local authorities across England and Wales.

# Load in dataset

df3 <- read_csv('../Data/GI_ethnic.csv')

Rows: 10592 Columns: 7
── Column specification ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Delimiter: ","
chr (4): Lower tier local authorities Code, Lower tier local authorities, Gender identity (4 categories), Ethnic group (8 categories)
dbl (3): Gender identity (4 categories) Code, Ethnic group (8 categories) Code, Observation

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

# Brief glimpse at underlying data structure
head(df3, 10)

Data Cleaning

• Clean column names
• Filter out unnecessary categories

Below, I provide another method ‘gsub()’ which can be used instead of the str_replace_all() method which has been demonstrated in the previous cleaning sections. Basically, looks for a pattern and applies the replacement to any column names which match the pattern.

# Remove all text within parentheses from column names and replace it with an empty string

# tilde operator (~) used to apply function 'gsub' to each colname
# .x represents each colname that gsub will be applied to
df3 <- df3 %>% 
  rename_with(~ gsub("\\s*\\([^)]*\\)", "", .x))

# Lowercase all text in column names and replace spaces with underscores
df3 <- df3 %>% 
  rename_with(~ tolower(gsub(" ", "_", .x)))

# Shorten the local authority column names as they are way too long
df3 <- df3 %>% 
  rename(LA_code = lower_tier_local_authorities_code,
         LA_name = lower_tier_local_authorities)

# Let's see if it worked
colnames(df3)

[1] "LA_code"              "LA_name"              "gender_identity_code" "gender_identity"      "ethnic_group_code"   
[6] "ethnic_group"         "observation"         

# Remove 'Does not apply' categories for the gender identity and ethnic group columns
df3 <- df3 %>% 
  filter(gender_identity_code != -8, ethnic_group_code != -8)

# Let's see if it worked.. 
unique(df3$gender_identity_code)

[1] 1 2 3

# Let's see if it worked..
unique(df3$ethnic_group_code)

[1] 1 2 3 4 5 6 7

Question

How does the rate of ‘non-response’ on gender identity vary among different ethnic groups across local authorities in England and Wales?

A subquestion this could help us answer:

Does the relationship between non-response and ethnic group % for local authorities differ between the ‘White’ categories and other ethnic groups?

Data pre-processing

Given that I want to explore the question above, I’d like to create a scatterplot which explores the relationship between the % of certain ethnic groups within local authorities and their non-response rates. Therefore, I’ll need to prep my x and y variables, so I’ll need to calculate the percentage of each ethnic group in each LA, and that ethnic groups non-response rate within each LA.

Calculate % of each ethnic group in each LA

# First, we're going to group our data by LA_name, ethnic group, and sum our observations
# This leaves us with the total of each ethnic group in each local authority
ethnic_totals <- df3 %>%
  group_by(LA_name, ethnic_group) %>%
  summarise(Ethnic_sum = sum(observation, na.rm = TRUE)) %>%
  ungroup()

`summarise()` has grouped output by 'LA_name'. You can override using the `.groups` argument.

# Print the first few rows to check
head(ethnic_totals)

# Calculate total observations for each local authority by grouping df3 by local authority and summing up obs
la_totals <- df3 %>%
  group_by(LA_name) %>%
  summarise(LA_sum = sum(observation, na.rm = TRUE)) %>%
  ungroup()

# Print the first few rows to check
head(la_totals)

# Merge the ethnic_totals and la_totals dataframes together
# by parameter specifies which column to perform merge on

grp_pct <- merge(ethnic_totals, la_totals, by = "LA_name")

# Calculate the percentage of each ethnic group within each local authority
# Store results in new column 

grp_pct <- grp_pct %>%
  mutate(Percentage = round((Ethnic_sum / LA_sum * 100), 2))

# Print the first few rows to check
head(grp_pct, 10)

Calculate Ethnic Group Non-Response Rates (%’s) Within LAs

# We already have our ethnic group totals which we can re-use...

ethnic_totals

# Calculate sum of non-responses for each ethnic group within each LA
# Filter df3 so that we only have non-response rows
# Group by LA and ethnic group then sum non-response obs and store the results in new column

non_response_totals <- df3 %>%
  filter(gender_identity == 'Not answered') %>%
  group_by(LA_name, ethnic_group) %>%
  summarise(NR_total = sum(observation, na.rm = TRUE)) %>%
  ungroup()

`summarise()` has grouped output by 'LA_name'. You can override using the `.groups` argument.

# Let's check it out.. 
head(non_response_totals)

# Merge the tethnic group totals with the ethnic group non-response totals
# c - used when we're referencing more than one column
# all.x - performs a left join
grp_nr <- merge(ethnic_totals, non_response_totals, by = c("LA_name", "ethnic_group"), all.x = TRUE)

# Let's check it out.. 

head(grp_nr)

# Calculate the non-response percentage for each ethnic group within each LA
# Store results in new column

grp_nr <- grp_nr %>%
  mutate(Eth_NR_Perc = round((NR_total / Ethnic_sum * 100), 2))

# Quick glance.. 
head(grp_nr)

Merge both datasets

Now that we’ve completed the necessary calculations, we are left with two datasets:

• grp_pct - details the % of each ethnic_group in each LA
• grp_nr - details the ethnic group non-response % in each LA

All we need to do now then, is merge these datasets together so that we can access the new columns and plot them:

• Percentage
• Eth_NR_Perc

# Merge the non-response data with the percentage of each ethnic group within each LA
# Use select to isolate columns I want to preserve in the merge, LA_sum is redundant...

nr <- merge(grp_nr, select(grp_pct, LA_name, ethnic_group, Percentage), by = c("LA_name", "ethnic_group"))

# Quick glance

head(nr)

Interactive scatterplot

In this section we’re going to:

1. Create a simple scatterplot exploring the relationship between the percentage of asian citizens within local authorities and their non-response rates

2. Implement a dropdown widget to update our scatterplot

# Subset dataframe so we only have responses from the asian ethnic group

asian <- nr %>%
  filter(ethnic_group == 'Asian, Asian British or Asian Welsh')

# Check it out.. 

head(asian)

# Initialize figure
fig3 <- plot_ly(data = asian,
               x = ~Percentage,
               y = ~Eth_NR_Perc,
               text = ~paste('LA Name:', LA_name,
                             '<br>Non-response Total:', NR_total,
                             '<br>Ethnic Group Total:', Ethnic_sum),
               hoverinfo = "text",
               mode = 'markers',  # Specify marker points
               type = 'scatter',  # Graph type - scatterplot
               name = 'Asian')  # Default visible graph


# Customize layout 
fig3 <- fig3 %>%
  layout(title = 'Non-Response Rates of the Asian Ethnic Group Across Local Authorities',
         xaxis = list(title = 'Percentage of Ethnic Group'),
         yaxis = list(title = 'Non-response Rate'),
         width = 700,
         height = 700)

Warning: Specifying width/height in layout() is now deprecated.
Please specify in ggplotly() or plot_ly()

# Show the plot
fig3

Dropdown selection

What we’re going to do now, is use Plotly’s ‘updatemenus’ in conjunction with the ‘update’ method to create a dropdown where we can switch between the Asian ethnic group, and the White ethnic group to make some comparisons.

Step 1: Initialise figure and add traces

We’ll start by creating a plot_ly figure with no data or variables specified. This is because we’re going to use add_trace to add our two sets of datapoints to the plot. ‘Traces’ refer to a set of data, so in our example we want to add a trace with the data points relating to our asian ethnic group, and another one for our white ethnic group. This will start to make sense when we look at the code below.

# Initialize a Plotly figure
fig4 <- plot_ly()

# Let's take a look..
# This is our building block
fig4

Warning: No trace type specified and no positional attributes specified
No trace type specified:
  Based on info supplied, a 'scatter' trace seems appropriate.
  Read more about this trace type -> https://plotly.com/r/reference/#scatter
No scatter mode specifed:
  Setting the mode to markers
  Read more about this attribute -> https://plotly.com/r/reference/#scatter-mode
Warning: No trace type specified and no positional attributes specified
No trace type specified:
  Based on info supplied, a 'scatter' trace seems appropriate.
  Read more about this trace type -> https://plotly.com/r/reference/#scatter
No scatter mode specifed:
  Setting the mode to markers
  Read more about this attribute -> https://plotly.com/r/reference/#scatter-mode

# Subset dataframe so we only have responses from the white ethnic group
white <- nr %>%
  filter(ethnic_group == 'White: English, Welsh, Scottish, Northern Irish or British')

# Quick check...
head(white)

# Add trace for the Asian ethnic group

fig4 <- fig4 %>% add_trace(
  data = asian,
  x = ~Percentage,
  y = ~Eth_NR_Perc,
  text = ~paste('LA Name:', LA_name,
                '<br>Non-response Total:', NR_total,
                '<br>Ethnic Group Total:', Ethnic_sum),
  type = 'scatter',
  mode = 'markers',
  name = 'Asian',
  hoverinfo = 'text',
  visible = T
)

# Add trace for the White ethnic group
fig4 <- fig4 %>% add_trace(
  data = white,
  x = ~Percentage,
  y = ~Eth_NR_Perc,
  text = ~paste('LA Name:', LA_name,
                '<br>Non-response Total:', NR_total,
                '<br>Ethnic Group Total:', Ethnic_sum),
  type = 'scatter',
  mode = 'markers',
  name = 'White',
  hoverinfo = 'text',
  visible = F
)

fig4

Step 2: Configure dropdown buttons and set initial visibility of datasets

# Define dropdown buttons for interactivity
fig4 <- fig4 %>% layout(
  title = "Non-Response Rates Across Local Authorities",
  xaxis = list(title = "Percentage of Ethnic Group"),
  yaxis = list(title = "Non-response Rate"),
  # Hide the legend, as interactive dropdown will handle trace visibility
  showlegend = FALSE,
  # Add dropdown menu for interactive plot updates
  updatemenus = list(
    list(
      type = "dropdown",
      buttons = list(
        list(
          # the update method changes plot attributes when a button is clicked
          method = "update",
          # First button makes Asian data visible and hides the White data
          args = list(list("visible" = list(TRUE, FALSE)),
                      # Update the title specific to the Asian data
                      list("title" = "Non-Response Rates of the Asian Ethnic Group Across Local Authorities")),
          # Specify button label
          label = "Asian"
        ),
        list(
          method = "update",
          args = list(list("visible" = list(FALSE, TRUE)),
                      list("title" = "Non-Response Rates of the White Ethnic Group Across Local Authorities")),
          label = "White"
        )
      )
    )
  )
)

# Display the figure
fig4

Dataset 3

This dataset includes sexual identity estimates by gender from 2010 to 2014. This is presented at a UK level, and broken down by England, Wales, Scotland and Northern Ireland. I wanted this guide to include a demo of how to make interactive line graphs with gender identity data, but unfortunately given this is only the first year that the ONS has collected this data that was not possible. So I found a dataset from 2015 which involves experimental statistics that have been used in the Integrated Household Survey. For more info, you can check out this ONS link.

# Load in dataset

df3 <- read_csv('../Data/cleaned_sexuality_df.csv')

Rows: 300 Columns: 5
── Column specification ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Delimiter: ","
chr (3): Country, Gender, Sexuality
dbl (2): Year, Percentage

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

# Brief glimpse at underlying data structure

head(df3, 10)

Data cleaning

When I first found this dataset it was very messy and formatted terribly, so I performed some cleaning on it in a separate jupyter notebook, to save cluttering this one and distracting from the main tutorial. If you’d like to see how I cleaned it up, please see the ‘Data_cleaning_sexuality.ipynb’ notebook.

Data pre-processing

The only pre-processing we’re going to do is subset our data by country, and also create 2 separate datasets for Gender = Men and Gender = Women. I’ll explain why this step is needed soon.

# Filter dataset to focus on England
england_df <- df3 %>%
  filter(Country == 'England')

# Let's check it worked.. 

unique(england_df$Country)

# Further filter data for each gender

men <- england_df %>% filter(Gender == "Men")
women <- england_df %>% filter(Gender == "Women")

# Let's check it worked

unique(men$Gender)

[1] "Men"

unique(women$Gender)

[1] "Women"

Interactive linegraph

Creating a simple line graph in plotly is pretty easy, but where plotly struggles (in R) is in handling facet plots. A facet plot is a type of visualisation that divides data into subplots based on categorical variables. What I’d like to do is create a facet plot of sexuality percentages in England (2010-2014) with individual subplots for our two genders. This is achieved easily in Python due to the plotly.express module, which provides a simple way to create facet plots. Unfortunately, we’ll have to go through a bit more of a longwinded route, where we’ll manually create our individual plots for each gender, then combine them using the subplot function. Also, plotly.express automatically manages legends to ensure they’re unified across facets, but R’s plotly requires that we manually sync up these legends. Womp womp. Let’s get to it.

# Create individual plot for each gender

# Create plots for each gender
men_plot <- plot_ly(men, 
                    x = ~Year, 
                    y = ~Percentage, 
                    color = ~Sexuality, 
                    type = 'scatter', 
                    # mode used to make sure our data points are connected by lines across the years
                    mode = 'lines+markers', 
                    hoverinfo = 'text',
                    text = ~paste("Year:", Year, "<br>Percentage:", Percentage, "<br>Sexuality:", Sexuality),
                    # legendgroup parameter ensures that data points relating to the same category are synced across plots
                    legendgroup = ~Sexuality,
                    # showlegend parameter set to TRUE only for this plot to avoid duplicate legends
                    showlegend = TRUE) %>%
  layout(xaxis = list(title = 'Year', tickvals = 2010:2014, ticktext = 2010:2014),
         yaxis = list(title = 'Percentage'),
         # Here we add an annotation to the graph to label the first subplot "Men"
         # Setting xref and yref to 'paper' simply means the annotation won't move if we zoom in or out
         annotations = list(
           list(x = 0.5, y = 1.05, text = "Men", showarrow = FALSE, xref='paper', yref='paper')))


women_plot <- plot_ly(women, 
                      x = ~Year, 
                      y = ~Percentage, 
                      color = ~Sexuality, 
                      type = 'scatter', 
                      mode = 'lines+markers', 
                      hoverinfo = 'text',
                      text = ~paste("Year:", Year, "<br>Percentage:", Percentage, "<br>Sexuality:", Sexuality),
                      legendgroup = ~Sexuality,
                      showlegend = FALSE) %>%
  layout(xaxis = list(title = 'Year', tickvals = 2010:2014, ticktext = 2010:2014),
         yaxis = list(title = 'Percentage'),
         annotations = list(
           list(x = 0.5, y = 1.05, text = "Women", showarrow = FALSE, xref='paper', yref='paper')))

# Let's take a look at one of these graphs

women_plot

# Combine individual plots using subplot
# Within subplot, define number of rows, make sure share same x axes and both axes titles
fig5 <- subplot(men_plot, women_plot, nrows = 2, shareX = TRUE, titleX = TRUE, titleY = TRUE) %>%
  layout(
    title = list(
      text = 'Sexuality Percentages by Gender in England (2010-2014)', 
      y = 0.98,  # Move the title higher up
      x = 0.5,   # Center the title
      xanchor = "center",
      yanchor = "top"
    ),
    margin = list(t = 100),  # Add space at the top for the title
    height = 800,
    width = 1000
  )

Warning: Specifying width/height in layout() is now deprecated.
Please specify in ggplotly() or plot_ly()

fig5

Sharing your interactive graphs online

I’m going to provide you first with a really simple way to host Plotly graphs specifically, then we’ll look into other more complex options that work with many visualisation packages.

1. Use Plotly’s ‘Chart Studio’. You can upload your visualisations directly from your coding environment and then get a link to share them online. You’ll need to sign up for an account but it’s free, unless you want to share the link privately then you’ll need to upgrade your account. Otherwise, for data that’s fine being out in the open, this is a good option.

2. Embed your graphs in GitHub pages.